Image pickup device and camera shake correction method

ABSTRACT

In an image pickup device, a controller includes a surplus area calculation unit to detect a surplus area for use in correcting camera shake based upon a size of an effective area on an image pickup surface and a size of an efficient area extracted in response to camera shake. Thus, this image pickup device can prevent a troublesome aspect in which camera shake correction reaches the correction end from occurring, even when a large camera shake that is beyond the surplus area occurs.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image pickup device for effectingcamera shake correction on an image signal and a camera shake correctionmethod.

2. Description of the Related Art

A cited patent reference 1 has disclosed so far a camera shakecorrection device comprising a pitching and yawing amount detectingmeans for detecting an amount in which an image is pitched or yawed dueto vibrations of a video camera body, a correction signal generatingmeans for generating a correction signal to correct pitching or yawingof an image based upon a detected output from the pitching and yawingamount detecting means, the correction range of the correction signalgenerating means being set freely variable, a correcting means operableso as to correct pitching or yawing of the image in response to thecorrection signal generated from the correction signal generating means,an operation range detecting means for detecting an operation range ofthe correcting means and a correction range setting means for variablysetting a correction range based upon a detected output from theoperation range detecting means.

The above-mentioned related-art image pickup device is able to shoot awide-angle image by extending an efficient area that is extracted whencamera shake is corrected. In that case, although the image in theefficient area has to be converted into an image with resolution of atelevision signal, it is possible to realize an image of wider-angle byextending the efficient area to the size of the whole area of the imagepickup surface.

[Cited Patent Reference 1]

Japanese laid-open patent application No. 7-327160

However, the above-mentioned related-art image pickup device isrequested to correct large camera shake as a magnification of lens isincreased. When a surplus area used to correct camera shake decreases asan image becomes a wide-angle image and large camera shake which isbeyond the surplus area occurs, there is then a disadvantage that atroublesome aspect in which camera shake correction reaches thecorrection end will occur. The reason for this is that the camera shakecorrection is suddenly limited at the correction end so that an imageextraction area is fixed.

In accordance with the cited patent reference 1, since the operationrange of the correction means is a linear integration output and thislinear integration output indicates an output obtained when a constantintegration coefficient is designated regardless of the size of thesurplus area, the linear integration output is limited at the same timeit reaches the correction end, and hence there is then a disadvantagethat a troublesome aspect in which an image suddenly reaches thecorrection end will occur.

SUMMARY OF THE INVENTION

In view of the aforesaid aspect, it is an object of the presentinvention to provide an image pickup device in which a troublesomeaspect in which an image reaches the correction end can be preventedfrom occurring even when a large camera shake signal which is beyond asurplus area is inputted.

It is another object of the present invention to provide an image pickupdevice in which a troublesome aspect caused when camera shake correctionis suddenly limited at the correction end by the input of a large camerashake signal can be decreased.

It is other object of the present invention to provide an image pickupdevice in which a stereotyped camera shake correction can be carried outat high speed.

It is a further object of the present invention to provide a camerashake correction method in which a troublesome aspect in which an imagereaches the correction end can be prevented from occurring even when alarge camera shake signal which is beyond a surplus area is inputted.

It is yet a further object of the present invention to provide a camerashake correction method in which a troublesome aspect caused when camerashake correction is suddenly limited at the correction end by the inputof large camera shake signal can be decreased.

It is still a further object of the present invention to provide acamera shake correction method in which a stereotyped camera shakecorrection can be carried out at high speed.

According to an aspect of the present invention, there is provided animage pickup device which is comprised of an image pickup means, acamera shake detecting means for detecting camera shake to provide acamera shake detection signal and a correcting means for correctingcamera shake of an image pickup signal obtained from the image pickupmeans by using the camera shake detection signal detected by the camerashake detecting means, wherein the correcting means includes a surplusarea detecting means for detecting a surplus area for use in camerashake correction based upon a size of an effective area on an imagepickup surface and a size of an efficient area extracted in response tocamera shake.

According to the present invention, the surplus area detecting meanscalculates a size of a current surplus area from a correction amountobtained from the last correction processing. The integrationcoefficient control means calculates an integration coefficient of acamera shake signal based upon a difference between a size of thesurplus area and a magnitude of an inputted camera shake signal.

Then, the thus calculated integration coefficient and the correctionamount of the last correction processing are multiplied, and the thusmultiplied result and the camera shake signal are integrated. Thus,camera shake of the image pickup signal can be corrected by using thethus calculated correction amount.

According to another aspect of the present invention, there is provideda camera shake correction method which is comprised of the steps of animage pickup step for obtaining an image pickup signal, a camera shakedetection step for detecting a camera shake detection signal and acorrection step for correcting camera shake of the image pickup signalobtained from the image pickup step by using the camera shake detectionsignal detected at the camera shake detection step, wherein thecorrection step includes a surplus area detection step for detecting asurplus area for use in camera shake correction based upon a size of aneffective area on an image pickup surface and a size of an efficientarea extracted in response to camera shake.

According to the present invention, the surplus area detection stepcalculates a size of a current surplus area from a correction amountobtained from the last correction processing. The integrationcoefficient control step calculates an integration coefficient of acamera shake signal based upon a difference between a size of thesurplus area and a magnitude of an inputted camera shake signal.

Then, the thus calculated integration coefficient and the correctionamount of the last correction processing are multiplied, and the thusmultiplied result and the camera shake signal are integrated. Thus,camera shake of the image pickup signal can be corrected by using thethus calculated correction amount.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram showing a camera shake correctiondevice for use with an image pickup device which is applied to theembodiment of the present invention;

FIG. 2 is a schematic block diagram showing an integration unit of acontroller for use with the camera shake correction device which isapplied to the embodiment of the present invention;

FIGS. 3A and 3B are respectively diagrams showing an effective area, anefficient area and a surplus area on an image pickup surface, whereinFIG. 3A shows the efficient area and the effective area on the imagepickup surface and FIG. 3B shows the surplus area on the image pickupsurface;

FIG. 4 is a diagram showing a relationship between an input signalinputted to an integration coefficient calculation unit and anintegration coefficient;

FIG. 5 is a flowchart to which reference will be made in explaining acamera shake correction control operation which is applied to theembodiment of the present invention;

FIG. 6 is a diagram showing compared results of a linear integrationoutput and a nonlinear integration output; and

FIG. 7 is a diagram showing a table of integration coefficients relativeto the surplus area of the image pickup surface.

DESCRIPTION OF THE PREFERRED EMBODIMENT

An image pickup device and a camera shake correction method according toan embodiment of the present invention will be described below withreference to the drawings.

[System Arrangement]

FIG. 1 of the accompanying drawings shows in block form a camera shakecorrection device for use with an image pickup device which is appliedto the embodiment of the present invention.

As shown in FIG. 1, an image pickup device 1 outputs an image of a wholearea (effective area) on an image pickup surface in the form of ananalog signal. An analog-to-digital converter (ADC) 2 converts theanalog signal outputted from the image pickup device 1 into a digitalsignal. A correction circuit 3 extracts an area (efficient area) fromone portion of the effective area in accordance with a correction amountoutputted from a controller 5. A camera shake detection unit 4 detectscamera shake applied to the image pickup device. The camera shakedetection unit 4 is able to detect camera shake by using a detectionmethod based upon a suitable means such as a gyro sensor and imagerecognition. The controller 5 calculates a correction amount forextracting the efficient area from the effective area based upon asurplus area for use in correcting camera shake and a camera shakesignal outputted from the camera shake detection unit 4 and outputs thethus calculated correction amount to the correction circuit 3.

FIG. 2 is a schematic block diagram showing an integration unit in thecontroller 5 of the camera shake correction device which is applied tothe embodiment of the present invention.

As shown in FIG. 2, a camera shake signal input 21 obtains the camerashake signal outputted from the camera shake detection unit 4 shown inFIG. 1. An integrator 22 integrates the camera shake signal input 21 anda signal from a multiplier 24. A correction output 23 outputs a signalintegrated by the integrator 22. A multiplier 24 multiplies anintegration coefficient obtained from an integration coefficientcalculation unit 26 with a signal from a delay device 25. The delaydevice 25 delays the signal of the correction output 23. The integrationcoefficient calculation unit 26 calculates an integration coefficientfor use with the multiplier 24 based upon a difference from a subtractor28 between a size of a surplus area obtained from a surplus areacalculation unit 27 and a magnitude of an inputted camera shake signal.The surplus area calculation unit 27 calculates a size of a surplusarea, which can be used to correct camera shake on the image pickupsurface, on the basis of a correction amount obtained through the delaydevice 25.

[Calculation of Surplus Area]

FIGS. 3A and 3B are schematic diagrams showing an effective area, anefficient area and a surplus area on an image pickup surface.

FIG. 3A shows the effective area, the efficient area, a camera shakevector and the surplus area on the image pickup surface.

Assuming now that an effective area 31 has a height ah and a width aw,that an efficient area 32 has a height bh and a width bw and that a lastcorrection vector is [vx⁻¹, vy⁻¹], then a surplus area 33 that can beused to correct camera shake is expressed by an area which is not madecommon to the effective area 31 and the efficient area 32 as shown inFIG. 3B. The sizes of the effective area 31 and efficient area 32 arenot fixed and may be changed dynamically in response to a suitablefactor such as a zoom magnification. At that time, in the area that canbe used to correct camera shake, a height sy of the side near thecorrection end can be expressed by the following equation (1) and awidth sx near the correction end can be expressed by the followingequation (2).sy=(ah−bh)/2−|vy ⁻¹|  (1)sx=(aw−bw)/2−|vx ⁻¹|  (2)[Calculation of Integration Coefficient]

FIG. 4 is a diagram showing a relationship between an integrationcoefficient calculation unit input and an integration coefficient.

As shown in FIG. 4, the integration coefficient monotonically increasesrelative to the integration coefficient calculation unit input. Assumingthat f represents a function that can satisfy the above-describedrelationship, tx represents a magnitude of an inputted camera shake inthe longitudinal direction, ty represents a magnitude of an inputtedcamera shake in the horizontal direction, ky represents an integrationcoefficient of the longitudinal direction and that kx represents anintegration coefficient of the horizontal direction, then theintegration coefficient ky of the longitudinal direction can beexpressed by the following equation (3) and the integration coefficientkx of the horizontal direction can be expressed by the followingequation (4):ky=f (sy−ty)  (3)kx=f (sx−tx)  (4)

If there is a large area that can be corrected, then the integrationcoefficient should be increased in order to increase efficiency ofcorrection. If on the other hand there is a small area that can becorrected, then the integration coefficient should be decreased in orderto decrease efficiency of correction. Under the circumstances in whichthe surplus length is short so that the integration coefficient willreach the correction end, a possibility in which the integrationcoefficient will reach the correction end can be decreased by loweringefficiency of correction.

[Calculation of Correction Amount]

Correction vector (vy0, vx0) of the longitudinal direction or thehorizontal direction can be calculated from the above-describedintegration coefficient. Assuming that (mvx, mvy) represents a camerashake motion vector, then the longitudinal direction correction vectorvy0 can be expressed by the following equation (5) and the horizontaldirection correction vector vx0 can be expressed by the followingequation (6):vy0=mvy+ky*vy ⁻¹  (5)vx0=mvx+kx*vx ⁻¹  (6)[Flowchart of Control Operation]

FIG. 5 is a flowchart to which reference will be made in explaining acamera shake correction control operation that is applied to theembodiment of the present invention.

This flowchart shown in FIG. 5 is read out from a memory (not shown)provided within the controller 5 each time a camera shake correctioncontrol operation is carried out.

Referring to FIG. 5, and following the start of operation, at a step S1,a size of a current surplus area is calculated from the last correctionamount. More specifically, the surplus area calculation unit 27 shown inFIG. 2 calculates the size of the current surplus area from the lastcorrection amount supplied from the delay device 25.

Then, control goes to the next step S2, whereat an integrationcoefficient of a camera shake signal is calculated from a differencebetween the size of the above-described surplus area and the magnitudeof the inputted camera shake. More specifically, the integrationcoefficient calculation unit 26 calculates the integration coefficientof the camera shake signal from the difference between the size of thesurplus area and the magnitude of the inputted camera shake.

Then, control goes to the next step S3, whereat the thus calculatedintegration coefficient and the last correction amount are multipliedwith each other. More specifically, the multiplier 24 multiplies thethus calculated integration coefficient with the last correction amount.

Then, control goes to the next step S4, whereat the above-describedmultiplied result and the camera shake signal are integrated. Morespecifically, the integrator 22 integrates the above-describedmultiplied result and the camera shake signal.

In this manner, the controller 5 calculates a correction amount and thecorrection circuit 3 corrects camera shake based upon the thuscalculated correction amount under control of the controller 5.

[Comparison of Integrated Outputs]

FIG. 6 is a diagram showing compared results obtained when a prior-artintegrated output and an inventive integrated output are compared witheach other.

As shown in FIG. 6, an integrated input 61 shows an integrated outputobtained when it is set to a DC component. A linear integrated output 62is obtained according to the prior-art system and shows an outputobtained when a constant integration coefficient is designatedregardless of the size of the surplus area. A nonlinear integratedoutput 63 is obtained according to the inventive system and shows anoutput obtained when the integration coefficient is dynamically changedin response to the size of the surplus area. At the same time the linearintegrated output 62 reaches a correction end 64, it is limited, andhence a troublesome aspect in which an image suddenly reaches thecorrection end 64 occurs. On the other hand, since efficiency ofcorrection is decreased before the nonlinear integrated output 63reaches the correction end 64, even when the nonlinear integrated output63 reaches the correction end 64 and is thereby limited, an image can beprevented from suddenly reaching the correction end 64.

FIG. 7 is a diagram showing a table of integration coefficient relativeto the surplus area. The table shown in FIG. 7 is calculated in advanceby the surplus area calculation unit 27 (see FIG. 2) and the integrationcoefficient calculation unit 26(see FIG. 2) and is then stored in amemory (not shown) provided within the controller 5.

As shown in FIG. 7, in the table, an integration coefficient 72 isoutputted as “1.00” when an integration coefficient calculation unitinput (pixel) of a surplus area 71 is “100”; the integration coefficient72 is outputted as “0.95” when the integration coefficient calculationunit input (pixel) of the surplus area 71 is “90”; the integrationcoefficient 72 is outputted as “0.90” when the integration coefficientcalculation unit input (pixel) of the surplus area is “80”; theintegration coefficient 72 is outputted as “0.85” when the integrationcoefficient calculation unit input (pixel) of the surplus area 71 is“70”; the integration coefficient 72 is outputted as “0.80” when theintegration coefficient calculation unit input (pixel) of the surplusarea 71 is “60”; the integration coefficient 72 is outputted as “0.70”when the integration coefficient calculation unit input (pixel) of thesurplus area 71 is “50”; the integration coefficient 72 is outputted as“0.60” when the integration coefficient calculation unit input (pixel)of the surplus area 71 is “40”; the integration coefficient 72 isoutputted as “0.50” when the integration coefficient calculation unitinput (pixel) of the surplus area 71 is “30”; the integrationcoefficient 72 is outputted as “0.40” when the integration coefficientcalculation unit input (pixel) of the surplus area 71 is “20”; theintegration coefficient 72 is outputted as “0.20” when the integrationcoefficient calculation unit input (pixel) of the surplus area 71 is“10”; and the integration coefficient 72 is outputted as “0.00” when theintegration coefficient calculation unit input (pixel) of the surplusarea 71 is “0”.

According to the embodiment of the present invention, since the size ofthe surplus area necessary for correcting camera shake is detected andthe integration coefficient used to integrate the camera shake signal ischanged in response to the difference between the size of the surplusarea and the magnitude of the input camera shake signal, it is possibleto decrease the troublesome aspect caused when the camera shakecorrection is suddenly limited at the correction end in response to theinput of the large camera shake signal.

According to the present invention, in an image pickup device comprisingan image pickup means, a camera shake detecting means for detectingcamera shake to provide a camera shake detection signal and a correctingmeans for correcting camera shake of an image pickup signal obtainedfrom the image pickup means by using the camera shake detection signaldetected by the camera shake detecting means, since the correcting meansincludes a surplus area detecting means for detecting a surplus area foruse in camera shake correction based upon a size of an effective area onan image pickup surface and a size of an efficient area extracted inresponse to camera shake, it is possible to decrease the troublesomeaspect caused when the camera shake correction is suddenly limited atthe correction end in response to the input of the large camera shakesignal, by detecting the size of the surplus area necessary forcorrecting camera shake and controlling the camera shake signal inresponse to the surplus area.

In the image pickup device according to the present invention, since thecorrecting means includes an integrating means for integrating thecamera shake detection signal linearly or nonlinearly and an integrationcoefficient control means for dynamically changing an integrationcoefficient used to integrate the camera shake detection signal inresponse to a difference between a size of the surplus area and amagnitude of the camera shake detection signal, it is possible todecrease the troublesome aspect caused when the camera shake correctionis suddenly limited at the correction end in response to the input ofthe large camera shake signal, by detecting the size of the surplus areanecessary for correcting camera shake and changing an integrationcoefficient in response to a difference between a size of the surplusarea and a magnitude of the camera shake detection signal.

In the image pickup device according to the present invention, since thecorrecting means includes a table having integration coefficientsrelative to sizes of the surplus area, a stereotyped camera shakecorrection processing can be carried out at high speed by using thesurplus areas and the integration coefficients stored in the memoryafter they have been calculated by the surplus area calculation unit andthe integration coefficient calculation unit.

According to the present invention, in a camera shake correction methodwhich is comprised of the steps of an image pickup step for obtaining animage pickup signal, a camera shake detection step for detecting acamera shake detection signal and a correction step for correctingcamera shake of the image pickup signal obtained from said image pickupstep by using the camera shake detection signal detected at, the camerashake detection step, since the correction step includes a surplus areadetection step for detecting a surplus area for use in camera shakecorrection based upon a size of an effective area on an image pickupsurface and a size of an efficient area extracted in response to camerashake, it is possible to decrease the troublesome aspect caused when thecamera shake correction is suddenly limited at the correction end inresponse to the input of the large camera shake signal, by detecting thesize of the surplus area necessary for correcting camera shake andcontrolling the camera shake signal in response to the surplus area.

Further, in the camera shake correction method according to the presentinvention, since the correction step includes an integration step forintegrating the camera shake detection signal linearly or nonlinearlyand an integration coefficient control step for dynamically changing anintegration coefficient used to integrate the camera shake detectionsignal in response to a difference between a size of the surplus areaand a magnitude of the camera shake detection signal, it is possible todecrease the troublesome aspect caused when the camera shake correctionis suddenly limited at the correction end in response to the input ofthe large camera shake signal, by detecting the size of the surplus areanecessary for correcting camera shake and changing an integrationcoefficient in response to a difference between a size of the surplusarea and a magnitude of the camera shake detection signal.

In the camera shake correction method according to the presentinvention, since the correction step corrects camera shake by using atable having integration coefficients relative to sizes of the surplusarea, a stereotyped camera shake correction processing can be carriedout at high speed by using the surplus areas and the integrationcoefficients stored in the memory after they have been calculated by thesurplus area calculation unit and the integration coefficientcalculation unit.

Having described a preferred embodiment of the invention with referenceto the accompanying drawings, it is to be understood that the inventionis not limited to that precise embodiment and that various changes andmodifications could be effected therein by one skilled in the artwithout departing from the spirit or scope of the invention as definedin the appended claims.

1. An image pickup device for use in a camera, the device comprising:image pickup means; camera shake detecting means for detecting a camerashake and outputting a camera shake detection signal; and correctingmeans for correcting a camera shake of an image pickup signal obtainedfrom said image pickup means by using said camera shake detectionsignal, wherein said correcting means includes surplus area detectingmeans for detecting a surplus area for use in camera shake correctionbased upon a size of an effective area on an image pickup surface of theimage pickup means and a size of an efficient area extracted in responseto the camera shake, integrating means for integrating said camera shakedetection signal and integration coefficient control means fordynamically changing an integration coefficient used to integrate saidcamera shake detection signal in response to a difference between a sizeof said surplus area and a magnitude of said camera shake detectionsignal.
 2. The image pickup device according to claim 1, wherein saidcorrecting means includes a table having a plurality of integrationcoefficients relative to a respective plurality of sizes of said surplusarea.
 3. A camera shake correction method comprising: an image pickupstep for obtaining an image pickup signal; a camera shake detection stepfor detecting a camera shake and outputting a camera shake detectionsignal; and a correction step for correcting a camera shake of saidimage pickup signal obtained from said image pickup step by using saidcamera shake detection signal, wherein said correction step includes asurplus area detection step for detecting a surplus area for use in acamera shake correction based upon a size of an effective area on animage pickup surface and a size of an efficient area extracted inresponse to the camera shake, an integration step for integrating saidcamera shake detection signal and an integration coefficient controlstep for dynamically changing an integration coefficient used tointegrate said camera shake detection signal in response to a differencebetween a size of said surplus area and a magnitude of said camera shakedetection signal.
 4. The camera shake correction method according toclaim 3, wherein said correction step corrects the camera shake by usinga table having a plurality of integration coefficients relative to arespective plurality of sizes of said surplus area.
 5. The image pickupdevice according to claim 1, in which said correcting means furtherincludes a multiplier for multiplying the integration coefficient with asignal corresponding to an output signal from the integrating means soas to obtain a multiplied signal and for supplying the multiplied signalto the integrating means and in which the integrating means integratessaid camera shake detection signal and the multiplied signal.
 6. Thecamera shake correction method according to claim 3, in which thecorrecting step further includes a multiplying step for multiplying theintegration coefficient with a signal corresponding to an output signalobtained from the integrating step so as to obtain a multiplied signaland in which the integrating step integrates said camera shake detectionsignal and the multiplied signal.